A Comparison of Checklist and Domain-Based Ratings in the Assessment of Objective Structured Clinical Examination (OSCE) Performance.

INTRODUCTION: The Objective Structured Clinical Examination (OSCE) is the mainstay of clinical assessment in the final-year undergraduate Family Medicine clerkship at King Faisal Specialist Hospital and Research Centre (KFSHRC). The gold standard for OSCE assessment is the checklist rating, completed by physician examiners. Numerous studies have suggested that global or domain-based OSCE ratings may be a better indicator of competence than checklist ratings. The aim of this study was to examine the utility of domain-based OSCE ratings in the context of final-year, undergraduate, Family Medicine OSCE examinations in Riyadh, Saudi Arabia. This is akin to an exercise in quality improvement, as we continuously look for ways to improve our OSCE assessment processes. METHODS: This study utilised a quantitative methodology. Three final year OSCE exams were chosen. Physicians rated each student using a checklist score and using a more holistic domain-based score. Physician checklist scores and physician domain-based scores were then compared, and correlation was assessed. We also looked at the internal consistency of the scoring methods. RESULTS: A significant correlation was found between checklist and domain-based scores by physicians for all exams (r=0.858, p<0.01), with a good internal consistency for these methodologies for all exams. CONCLUSION: The results demonstrate that both checklist and domain-based scores offer some benefit to the assessment, with a similar internal consistency and strong correlation. Domain-based ratings should be utilised for softer skills that are not easily assessed by checklists. There is clearly a need to rethink our OSCE assessment. The assessment should combine checklist and domain-based physician scores. As trainees become more experienced, checklist OSCE may penalise directness and efficiency, while domain-based ratings would offer a better appraisal of competence, and have been shown to be more sensitive to the level of training and expertise. Changing the assessment methods will lead to necessary changes in the student approach to the OSCE and improve authenticity and validity.

Patient ratings in exercise therapy for the management of tendinopathy: a systematic review with meta-analysis.

OBJECTIVE: To synthesise exercise therapy intervention data investigating patient rating outcomes for the management of tendinopathy. DESIGN: A systematic review and meta-analysis of randomized controlled trials investigating exercise therapy interventions and reporting patient rating outcomes. SETTING: Any setting in any country listed as very high on the human development index. PARTICIPANTS: People with a diagnosis of any tendinopathy of any severity or duration. INTERVENTIONS: Exercise therapy for the management of tendinopathy comprising five different therapy classes: 1) resistance; 2) plyometric; 3) vibration; 4) flexibility, and 5) movement pattern retraining modalities, were considered for inclusion. MAIN OUTCOME MEASURES: Outcomes measuring patient rating of condition, including patient satisfaction and Global Rating of Change (GROC). RESULTS: From a total of 124 exercise therapy studies, 34 (Achilles: 41%, rotator cuff: 32%, patellar: 15%, elbow: 9% and gluteal: 3%) provided sufficient information to be meta-analysed. The data were obtained across 48 treatment arms and 1246 participants. The pooled estimate for proportion of satisfaction was 0.63 [95%CrI: 0.53-0.73], and the pooled estimate for percentage of maximum GROC was 53 [95%CrI: 38-69%]. The proportion of patients reporting positive satisfaction and perception of change increased with longer follow-up periods from treatment onset. CONCLUSION: Patient satisfaction and GROC appear similar and are ranked moderately high demonstrating that patients generally perceive exercise therapies positively. Further research including greater consistency in measurement tools is required to explore and where possible, identify patient- and exercise-related moderating factors that can be used to improve person-centred care. SYSTEMATIC REVIEW REGISTRATION NUMBER: PROSPERO ID=CRD42020168187 CONTRIBUTION OF PAPER.

Patient-Reported Outcomes at Return to Sport After Lateral Ankle Sprain Injuries: A Report From the Athletic Training Practice-Based Research Network.

CONTEXT: Limited evidence exists regarding the assessment of single-item patient-reported outcomes when patients are medically cleared to return to sport after a lateral ankle sprain (LAS) injury. OBJECTIVE: To evaluate self-reports of improvement in health status, pain, function, and disability at return to sport after an LAS. DESIGN: Descriptive study. SETTING: Sixty-nine athletic training facilities across 24 states. PATIENTS OR OTHER PARTICIPANTS: A total of 637 patients (males = 53.2%) who were diagnosed with an LAS, restricted from sport after injury, and subsequently medically cleared to return to sport within 60 days were included. MAIN OUTCOME MEASURE(S): Descriptive statistics were used to summarize scores for health status (Global Rating of Change), pain (Numeric Pain Rating Scale), function (Global Rating of Function), and disability (Global Rating of Disability). Mann-Whitney U tests were used to compare score differences between sexes. A Kaplan-Meier analysis was performed to provide a visual depiction of sex differences in the time to return to sport. RESULTS: Most patients sustained an LAS injury while participating in basketball, football, or soccer and were cleared to return to sport 8 days after injury. More than two-thirds of patients reported a meaningful improvement in health status between the time of injury and return to sport. However, many noted deficits related to pain (65.1%), function (86.2%), or disability (35.8%) at return to sport. No differences were seen between males and females for pain (P = .90), function (P = .68), change in health status (P = .45), or disability (P = .21) at return to sport, although males returned to sport slightly sooner than females (P = .025). CONCLUSIONS: Despite self-perceived improvements in health status since the time of injury, patients typically returned to sport with deficits in pain, function, and disability after an LAS. Patients may be returning to unrestricted sport participation before they feel their bodies have fully recovered from the injury.

Recovery Effects of Foam Rolling on Psychophysiological Responses in Thai National Male Ice Hockey Players.

We examined the short-term effects of foam rolling (FR), dynamic stretching (DS), and passive rest (PR) following simulated ice hockey exercise (IHE) on heart rate (HR), blood lactate (BL), leg choice reaction time (CRTleg), rating of perceived exertion (RPE), and global rating of change (GRC) in elite ice hockey players. The study followed a randomized cross-over design. Fifteen national male ice hockey players were assigned to the FR, DS, or PR interventions for 10 mins following 35- min of simulated IHE. HR and BL were obtained at 0-, 5- and 10-min post-intervention. CRTleg and RPE were assessed pre-and post-intervention. GRC was evaluated post-intervention. The PR decreased HR faster than the DS at 5-min of post-treatment. Whereas the FR and DS reduced BL levels faster than the PR at 5- and 10-min post-treatment. There was no difference in CRTleg among the FR, DS, and PR. The FR had lower RPE scores compared to the DS and PR post-treatment. As perceptual aspects, the FR was the most preferred treatment by ice hockey athletes. The FR and DS exerted more beneficial effects on BL but not on HR by the passive rest. The FR showed the most effective treatment on the psychological demands by improving RPE and perceptual responses over the DS and PR. Thus, the FR could be used as a choice for post-game recovery treatment on improving physiological and perceptual responses following an intense match-play in ice hockey players.

Corrigendum: Evaluation of the Effectiveness of the Tap Test by Combining the Use of Functional Gait Assessment and Global Rating of Change.

[This corrects the article DOI: 10.3389/fneur.2022.846429.].

Assessing technical skill in ankle fracture surgery from the postoperative radiograph : pilot development and validation of a final product analysis core outcome set.

AIMS: To identify a core outcome set of postoperative radiographic measurements to assess technical skill in ankle fracture open reduction internal fixation (ORIF), and to validate these against Van der Vleuten's criteria for effective assessment. METHODS: An e-Delphi exercise was undertaken at a major trauma centre (n = 39) to identify relevant parameters. Feasibility was tested by two authors. Reliability and validity was tested using postoperative radiographs of ankle fracture operations performed by trainees enrolled in an educational trial (IRCTN 20431944). To determine construct validity, trainees were divided into novice (performed < ten cases at baseline) and intermediate groups (performed ≥ ten cases at baseline). To assess concurrent validity, the procedure-based assessment (PBA) was considered the gold standard. The inter-rater and intrarater reliability was tested using a randomly selected subset of 25 cases. RESULTS: Overall, 235 ankle ORIFs were performed by 24 postgraduate year three to five trainees during ten months at nine NHS hospitals in England, UK. Overall, 42 PBAs were completed. The e-Delphi panel identified five 'final product analysis' parameters and defined acceptability thresholds: medial clear space (MCS); medial malleolar displacement (MMD); lateral malleolar displacement (LMD); tibiofibular clear space (TFCS) (all in mm); and talocrural angle (TCA) in degrees. Face validity, content validity, and feasibility were excellent. PBA global rating scale scores in this population showed excellent construct validity as continuous (p < 0.001) and categorical (p = 0.001) variables. Concurrent validity of all metrics was poor against PBA score. Intrarater reliability was substantial for all parameters (intraclass correlation coefficient (ICC) > 0.8), and inter-rater reliability was substantial for LMD, MMD, TCA, and moderate (ICC 0.61 to 0.80) for MCS and TFCS. Assessment was time efficient compared to PBA. CONCLUSION: Assessment of technical skill in ankle fracture surgery using the first postoperative radiograph satisfies the tested Van der Vleuten's utility criteria for effective assessment. 'Final product analysis' assessment may be useful to assess skill transfer in the simulation-based research setting. Cite this article: Bone Jt Open 2022;3(6):502-509.

Evaluation of the Effectiveness of the Tap Test by Combining the Use of Functional Gait Assessment and Global Rating of Change.

Background: Although the tap test for patients with suspected idiopathic normal pressure hydrocephalus (iNPH) is still often performed as part of the preoperative evaluation, it is true that some studies have reported the limitations of the tap test, claiming that it does not provide the additional information for appropriate patient selection for surgery. We aimed to determine whether a better method of pre- and post-tap test assessment could lead to appropriate patient selection for shunting. Methods: We performed the tap test as part of the preoperative evaluation in all 40 patients who underwent lumboperitoneal shunt surgery for iNPH from April 2021 to September 2021. We retrospectively analyzed the patient data. We examined whether a comprehensive evaluation of the effect of the tap test using the Functional Gait Assessment (FGA) and Global Rating of Change (GRC) scales would identify a wider range of patients who would benefit from shunt surgery than the 3-m Timed Up and Go test (TUG) alone. Results: Assuming a prevalence of 1% for iNPH, the TUG had a sensitivity of 0.23, specificity of 0.71, positive likelihood ratio of 0.79, and negative likelihood ratio of 1.09. When improvement in either the FGA or the GRC was used as a criterion for the validity of the tap test, the sensitivity was 0.88, specificity was 0.17, positive likelihood ratio was 1.06, and negative likelihood ratio was 0.71. Conclusion: Improvement in either the FGA or the GRC is a more sensitive criterion for the effectiveness of the tap test for the gait aspect than the TUG. Since the negative likelihood ratio is lower than that for the TUG alone, it is more appropriate to exclude patients with neither FGA nor GRC improvement from surgical indications than to exclude surgical indications based on a negative TUG.

A resource efficient and reliable standard setting method for OSCEs: Borderline regression method using standardized patients as sole raters in clinical case encounters with medical students.

Finding a reliable, practical and low-cost criterion-referenced standard setting method for performance-based assessments has proved challenging. The borderline regression method of standard setting for OSCEs has been shown to estimate reliable scores in studies using faculty as raters. Standardized patients (SPs) have been shown to be reliable OSCE raters but have not been evaluated as raters using this standard setting method. Our study sought to find whether SPs could be reliably used as sole raters in an OSCE of clinical encounters using the borderline regression standard setting method.SPs were trained for on a five-point global rating scale. In an OSCE for medical students, SPs completed skills checklists and the global rating scale. The borderline regression method was used to create case passing scores. We estimated the dependability of the final pass or fail decisions and the absolute dependability coefficients for global ratings, checklist scores, and case pass-score decisions using generalizability theory.The overall dependability estimate is 0.92 for pass or fail decisions for the complete OSCE. Dependability coefficients (0.70-0.86) of individual case passing scores range demonstrated high dependability.Based on our findings, the borderline regression method of standard setting can be used with SPs as sole raters in a medical student OSCE to produce a dependable passing score. For those already using SPs as raters, this can provide a practical criterion-referenced standard setting method for no additional cost or faculty time.

Validity and reliability of global ratings of satisfaction with epilepsy surgery.

OBJECTIVE: We aimed to assess the reliability and validity of single-item global ratings (GR) of satisfaction with epilepsy surgery. METHODS: We recruited 240 patients from four centers in Canada and Sweden who underwent epilepsy surgery ≥1 year earlier. Participants completed a validated questionnaire on satisfaction with epilepsy surgery (the ESSQ-19), plus a single-item GR of satisfaction with epilepsy surgery twice, 4-6 weeks apart. They also completed validated questionnaires on quality of life, depression, health state utilities, epilepsy severity and disability, medical treatment satisfaction and social desirability. Test-retest reliability of the GR was assessed with the intra-class correlation coefficient (ICC). Construct and criterion validity were examined with polyserial correlations between the GR measure of satisfaction and validated questionnaires and with the ESSQ-19 summary score. Non-parametric rank tests evaluated levels of satisfaction, and ROC analysis assessed the ability of GRs to distinguish among clinically different patient groups. RESULTS: Median age and time since surgery were 42 years (IQR 32-54) and 5 years (IQR 2-8), respectively. The GR demonstrated good to excellent test-retest reliability (ICC = 0.76; 95% CI 0.67-0.84) and criterion validity (0.85; 95% CI 0.81-0.89), and moderate correlations in the expected direction with instruments assessing quality of life (0.59; 95% CI 0.51-0.63), health utilities (0.55; 95% CI 0.45-0.65), disability (-0.51; 95% CI -0.41, -0.61), depression (-0.48; 95% CI -0.38, -0.58), and epilepsy severity (-0.48; 95% CI -0.38, -0.58). As expected, correlations were lower for social desirability (0.40; 95% CI 0.28-0.52) and medical treatment satisfaction (0.33; 95% CI 0.21-0.45). The GR distinguished participants who were seizure-free (AUC 0.75; 95% CI 0.67-0.82), depressed (AUC 0.75; 95% CI 0.67-0.83), and self-rated as having more severe epilepsy (AUC 0.78; 95% CI 0.71-0.85) and being more disabled (AUC 0.82; 95% CI 0.74-0.90). SIGNIFICANCE: The GR of epilepsy surgery satisfaction showed good measurement properties, distinguished among clinically different patient groups, and appears well-suited for use in clinical practice and research.

Validation of Task-Specific Rating Scale for Open Balloon Catheter Arterial Embolectomy: An Assessor-Blinded Quasi-Experimental Pilot Study.

Objective: To develop and validate a task-specific rating scale (TSRS) by comparing with the Global Rating Scale (GRS) for the evaluation of brachial artery embolectomy (BAE). Methods: Participants were divided into expert and novice groups who were oriented on the locally developed simulator model. The following day, an embolectomy procedure was performed independently by the participants and graded by two independent assessors using the GRS and TSRS. Validity was evaluated using Pearson's correlation coefficient (r), reliability by the interclass correlation coefficient (ICC), and agreement by Bland-Altman plots. A p-value <0.05 was considered significant. Results: Thirty-two participants were enrolled in this study. The overall TSRS was found to be a valid assessment tool (r=0.82; 95% confidence interval [CI]: 0.66, 0.91; p<0.001). Domain-specific analyses showed a moderate positive association between all domains (p<0.05), except for instrument handling (r=0.09; 95%CI: -0.27, 0.42; p=0.642). The ICC for overall scores showed excellent reliability for both instruments, GRS and TSRS, with values of 0.97 and 0.92, respectively. Conclusion: The TSRS was found to be a valid and reliable assessment tool for BAE; however, for some domains, such as instrument handling and time and motion, it has limited reliability.

Mastering microsurgery: A novel benchmarking tool for microsurgical training.

INTRODUCTION AND OBJECTIVE: The shift of surgical training from apprenticeship model towards competency-based training requires objective assessments of microsurgical skills to achieve mastery. Our objective was to create a novel platform to provide feedback to surgical trainees relative to competency expected at their level of exposure. METHODS: A 5-day simulated microsurgery course was run between 2013 and 2016 with 118 participants. Video recordings of end-to-end micro-anastomoses were collected on days 1, 3, and 5, along with analysis of hand motion . Videos were assessed to calculate the QMUL Global Rating Scale (QMUL GRS). Two charts were created to track the acquisition of microsurgical skills. One to plot the number of hand movements and the other for QMUL GRS, against the participants' cumulative number of micro-anastomoses performed. Participant data were used to calculate a skills acquisition line for each quartile, analogous to percentiles on a growth chart. RESULTS: For a cumulative number of 0 prior micro-anastomoses, the GRS score at the 25th, 50th, and 75th percentile was 45.5, 55, and 62, respectively. As the number of cumulative anastomoses increased, there was a distinct increase in the GRS score to 89.5, 93, and 92 for the group with 55-100 previous micro-anastomoses. This was in keeping with a decreased number of hand movements with increasing experience. CONCLUSION: In conclusion, our tool allows trainees to track where they lie in the skill spectrum relative to prior experience which permits the provision of focused training to trainees at lower percentiles, which has potential to translate to improved clinical outcomes.

Microsurgical Anastomosis Rating Scale (MARS10): A Final Product Scoring System for Initial Microsurgical Training.

BACKGROUND: High cost and ethical controversy of using living models in microsurgical training made non-living models more popular. However, non-living models don't provide appropriate feedback of microsurgical performance. Currently existing Global Rating Scales used for advanced microsurgical skills validation are difficult to apply on non-living model. This study presents a simple instrument for basic assessment of microsurgical anastomosis on non-living model. METHODS: Seventy medical students were divided into 2 groups depending on their prior microsurgical experience. Each participant performed 3 end-to-end anastomoses on chicken femoral artery model. Anastomoses were reviewed by 3 blinded experts and then photographed. Evaluation included a patency tests, longitudinal cut of anastomosis, and the newly proposed tool 10 Point Microsurgical Anastomosis Rating Scale (MARS10). Presented scale consists of 5 factors important for anastomosis closure (anastomosis closure, suture spacing, bites size, knot tying, and cut ends length), graded on 3 point scale (0-2 points). Results were analyzed with analysis of variance, Spearman correlation, and t Student test. RESULTS: Anastomoses evaluated by experts as patent significantly correlated with a high summary score in MARS10 scale (r = 0.73 P < .0001). There was a significant difference in MARS10 score between groups (P < .0001). There were no significant inter-rater differences in scoring among all 3 evaluators (p > .05). CONCLUSIONS: 10 Point Microsurgical Anastomosis Rating Scale is a quick, valid, and reliable tool to assess microsurgical end-to-end arterial anastomoses on non-living model.

HISTORIQUE: Le coût élevé et la controverse éthique liés à l'utilisation de modèles vivants pendant la formation en microchirurgie ont popularisé l'utilisation des modèles non vivants. Cependant, les modèles non vivants ne donnent pas de commentaires appropriés sur l'exécution de la microchirurgie. Les échelles d'évaluation mondiales actuellement utilisées pour valider les habiletés microchirurgicales avancées sont difficiles à appliquer sur des modèles non vivants. La présente étude propose un simple instrument d'évaluation fondamentale de l'anastomose microchirurgicale sur un modèle non vivant. MÉTHODOLOGIE: Les chercheurs ont divisé 70 étudiants en médecine en deux groupes d'après leur expérience microchirurgicale. Chaque participant a effectué trois anastomoses bout à bout sur un modèle d'artère fémorale de poulet. Trois experts ont examiné les anastomoses en aveugle puis les ont photographiées. L'évaluation incluait un test de perméabilité, la coupe longitudinale de l'anastomose et la nouvelle échelle d'évaluation de l'anastomose microchirurgicale en 10 points (MARS10). Cette échelle est composée de cinq facteurs importants pour fermer l'anastomose (fermeture de l'anastomose, espace entre les sutures, dimension des piqûres d'aiguille, exécution des nœuds et longueur des extrémités de fils), classés sur une échelle de trois points (de 0 à 2). Les chercheurs ont analysé les résultats avec l'analyse de variance, la corrélation de Spearman et le test de Student. RÉSULTATS: Les anastomoses évaluées comme perméables par les experts étaient corrélées par un sommaire de cotation élevé dans l'échelle de MARS10 (r = 0,73 P < 0,0001). Il y avait une différence significative du score MARS10 entre les groupes (p < 0,0001). Il n'y avait pas de différences interévaluateur significatives dans les scores des trois évaluateurs (p > 0,05). CONCLUSIONS: L'échelle de MARS10 est un outil rapide, valide et fiable pour évaluer les anastomoses artérielles microchirurgicales bout à bout sur des modèles non vivants.

Análisis de la aceptación y validez de los métodos utilizados para la implementación de un programa de formación basado en competencias en un servicio de Medicina Intensiva de un hospital universitario de referencia / Acceptance and validity of the methods used to implement a competency based medical education programme in an Intensive Care Department of a teaching referral center

Objetivos: 1) Determinar la satisfacción de tutores y residentes con la metodología utilizada para la implementación de CoBaTrICE, y 2) determinar la validez y la fiabilidad de las escalas de valoración global diseñados ad hoc para analizar el desempeño de los residentes con fines formativos. Diseño: Estudio prospectivo de cohortes. Participantes: Todos los residentes y tutores del Servicio de Medicina Intensiva del Hospital Universitario y Politécnico La Fe de Valencia. Intervención: En marzo del 2016 se inició la implementación de CoBaTrICE sustentada en: 1) formación de los tutores en técnicas de retroalimentación; 2) realización por los residentes de múltiples ejercicios reales de evaluación objetiva y estructurada para adquirir las competencias del programa, y 3) uso de un portafolio electrónico para registrar las evidencias del progreso y estimular la reflexión. Métodos: La satisfacción con CoBaTrICE se evaluó mediante una encuesta realizada tras 9 meses de implementación a los 15 residentes y 5 tutores del servicio. Se preguntó sobre la metodología de las evaluaciones, calidad de la retroalimentación, autorregulación del aprendizaje y utilidad del portafolio. Se determinaron la consistencia interna (alfa de Cronbach), índices de generalizabilidad y fiabilidad interjueces (índice de correlación intraclase) de las escalas de valoración global. Resultados: La aplicación de CoBaTrICE fue satisfactoria en todas las dimensiones estudiadas. Se constataron la validez y la fiabilidad de las escalas de valoración utilizadas. Conclusiones: La metodología utilizada para implementar CoBaTrICE fue valorada positivamente por tutores y residentes. Las escalas de valoración global utilizadas en la evaluación formativa demostraron ser válidas, fiables y reproducibles. (AU)

Objectives; 1. To determine the satisfaction of tutors and residents with a specific methodology used to implement CoBaTrICE; 2. To determine the reliability and validity of the global rating scales designed ad hoc to assess the performance of the residents for training purposes. Design Prospective cohort study Participants All the residents and tutors of the ICU Department of the Hospital Universitario y Politécnico la Fe de Valencia. Intervention CoBaTrICE implementation started in March 2016, it was based on: 1) Training the tutors in feedback techniques; 2) Performing multiple objective and structured work based assessments to achieve the competences of the program; and 3) The use of an electronic portfolio to promote learning reflection and to collect the evidence that learning was taking place. Methods The acceptance of CoBaTrICE was explored through a satisfaction survey conducted after 9 months of implementation of the training program. The 15 residents and 5 tutors of the ICU Department were asked about the methodology of the formative assessments, the quality of the feedback, self-learning regulation and the electronic portfolio usefulness. The validity of the global rating scales was assessed through the tests alfa de Cronbach, reliability and generalizability indexes, and intraclass correlation coefficient. Results The implementation of CoBaTrICE was satisfactory in all the dimensions studied. The global rating scales used for formative purposes showed reliability and validity. Conclusions The methodology used to implement CoBaTrICE was highly valued by tutors and residents. The global rating scales used for formative purposes showed reliability and validity. (AU)

Acceptance and validity of the methods used to implement a competency based medical education programme in an Intensive Care Department of a teaching referral center.

OBJECTIVES: 1. To determine the satisfaction of tutors and residents with a specific methodology used to implement CoBaTrICE. 2. To determine the reliability and validity of the global rating scales designed ad hoc to assess the performance of the residents for training purposes. DESIGN: Prospective cohort study. PARTICIPANTS: All the residents and tutors of the ICU Department of the Hospital Universitario y Politécnico la Fe de Valencia. INTERVENTION: CoBaTrICE implementation started in March 2016, it was based on: (1) Training the tutors in feedback techniques; (2) Performing multiple objective and structured work based assessments to achieve the competences of the program; and (3) The use of an electronic portfolio to promote learning reflection and to collect the evidence that learning was taking place. METHODS: The acceptance of CoBaTrICE was explored through a satisfaction survey conducted after 9 months of implementation of the training program. The 15 residents and 5 tutors of the ICU Department were asked about the methodology of the formative assessments, the quality of the feedback, self-learning regulation and the electronic portfolio usefulness. The validity of the global rating scales was assessed through the tests alfa de Cronbach, reliability and generalizability indexes, and intraclass correlation coefficient. RESULTS: The implementation of CoBaTrICE was satisfactory in all the dimensions studied. The global rating scales used for formative purposes showed reliability and validity. CONCLUSIONS: The methodology used to implement CoBaTrICE was highly valued by tutors and residents. The global rating scales used for formative purposes showed reliability and validity.

Associations between physicians' global assessment of disease activity and patient-reported outcomes in patients with systemic lupus erythematosus: A longitudinal study.

OBJECTIVE: To determine longitudinal associations between Physician Global Assessment (PGA) and patient-reported outcomes (PROs) in patients with systemic lupus erythematosus (SLE). METHODS: Patients attending a rheumatology clinic between 2013 and 2017 completed specific (SLEQOL) and generic (SF36) health-related quality of life (HRQoL) surveys and rated their global rating of change (GRC) at each visit. PGA, SLEDAI-2K and SLE Flare Index (SFI) were also captured on all visits. Generalised estimating equations (GEE) methods were used to examine longitudinal associations of PGA with PROs and clinical indicators. RESULTS: 337 patients were followed for a median [IQR] of 3.2 [1.6, 3.4] years (2,059 visits). High PGA (>1) was strongly associated with high SLEDAI-2K scores, the presence of flares and poor PROs. Odd ratios (OR) [95% CI] of PGA > 1 in patients with SLEDAI-2K >4 & <10 and SLEDAI-2K ≥10, compared to SLEDAI-2K ≤ 4, were 3.46 [2.36, 5.08], p < 0.001 and 6.39 [4.30, 9.49], p < 0.001, respectively. OR [95% CI] of PGA > 1 in patients with mild-to-moderate or severe flares were 2.09 [1.62, 2.71], p < 0.001 and 4.42 [3.21, 6.07], p < 0.001, respectively. Mental components of both SLEQOL (mood, self-image) and SF36 (MCS) surveys demonstrated significant associations with high PGA. After adjusting for SLEDAI-2K, one-point increase in PGA was associated with reductions in SLEQOL total score and SF36-MCS by 2.33 (regression coefficient (RC) [95% CI] = -2.33 [-3.77, -0.88], p = 0.002), and 4.16 (RC [95% CI] = -4.16 [-5.19, -3.13], p < 0.001) points, respectively. Associations of some physical components (SLEQOL-symptoms, and SF36-PCS) with PGA attenuated when adjusted for SLEDAI-2K. Patients who rated low scores of GRC, which indicate health deterioration, were twice as likely to have PGA > 1 (OR [95%CI] 1.99 [1.25, 3.16], p = 0.004). CONCLUSION: High PGA was strongly associated with poor mental health, high disease activity and flares. This study confirms the value of PGA as an efficient assessment tool for SLE.

Training Surgeons to Perform Arthroscopic All-Inside Meniscal Repair: A Randomized Controlled Trial Evaluating the Effectiveness of a Novel Cognitive Task Analysis Teaching Tool, Imperial College London/University College London Meniscus Repair Cognitive Task Analysis (IUMeRCTA).

BACKGROUND: All-inside meniscal repair is an increasingly common technique for the surgical treatment of meniscal tears. There are currently no standardized techniques for training residents in this procedure. Cognitive task analysis (CTA) is a method of analyzing and standardizing key steps in a procedure that allows training to be conducted in a validated and reproducible manner. PURPOSE: (1) To design a digital CTA teaching tool for a standardized all-inside meniscal repair. (2) To evaluate whether CTA-trained residents would perform better in a meniscal repair task compared with a control group who underwent traditional apprenticeship methods of training. STUDY DESIGN: Controlled laboratory study. METHODS: Three expert knee surgeons were interviewed using a modified Delphi method to generate a consensus among the ideal technical steps, cognitive decision points, and common errors and solutions for an all-inside meniscal repair. This written information was then combined with visual and audio components and integrated onto a digital platform to create the Imperial College London/University College London Meniscus Repair Cognitive Task Analysis (IUMeRCTA) tool. Eighteen novice residents were randomized into an intervention group (digital CTA tool) and control group (equipment instruction manual). Both groups performed an all-inside meniscal repair on high-fidelity, phantom knee models and were assessed by expert surgeons, blinded to the interventions, using a validated global rating scale (GRS). After a power calculation, median GRS scores were compared between groups using the Mann-Whitney U test; significance was set at P < .05. RESULTS: For the IUMeRCTA tool design, the procedure was divided into 55 steps across 9 phases: (1) preoperative planning, (2) theater and patient setup, (3) portal placement, (4) meniscal examination, (5) tear reduction, (6) suture planning, (7) suture insertion, (8) repair completion, and (9) postoperative care and rehabilitation. For the trial, the intervention group (mean ± SD GRS, 32 ± 2.9) performed significantly better than did the control group (GRS, 24 ± 3.3; P < .001). CONCLUSION: This is the first CTA tool to demonstrate objective benefits in training novices to perform an arthroscopic all-inside meniscal repair. CLINICAL RELEVANCE: The IUMeRCTA tool is an easily accessible and effective adjunct to traditional teaching that enhances learning the all-inside meniscal repair for novice surgeons.

The assessment of researchers' competence in experimental procedures with laboratory animals: A three-step methodology to develop a global rating scale.

To conduct animal experiments, researchers must be competent to handle and perform interventions on living animals in compliance with regulations. Laboratory animal science training programmes and licensing bodies therefore need to be able to reliably ensure and certify the professional competence of researchers and technicians. This requires access to assessment strategies which can verify knowledge as well as capturing performative and behavioural dimensions of assessment. In this paper, we describe the process of developing different global rating scales measuring candidates' competence in a performative assessment. We set out the following sequence, with three crucial phases, in the process of scale development: (a) Item Development, (b) Scale Development and (c) Piloting of the Scale. We note each phase's different sub-steps. Despite the emergent need to ensure the competence of researchers using animals in scientific procedures, to our best knowledge there are very few species and procedure/skill specific assessment tools for this purpose, and the assessment methodology literature in the field is very limited. This paper provides guidance for those who need to develop and assess proficiency in laboratory animal procedures by setting out a method that can be used to create the required tools and illustrating how competence assessment strategies can be implemented.

Impact of cumulative experience on the quality of screening colonoscopy: A 13-year observational study.

OBJECTIVE: To investigate trends in quality of screening colonoscopy (using the Global Rating Score) in the 13 years since introduction of the Bowel Cancer Screening Programme in England. SETTING: An English Bowel Cancer Screening Programme colonoscopy service from 2007 to 2019. METHODS: A retrospective observational study was undertaken using a prospectively collected database in order to analyse trends in screening endoscopies (including patients following positive faecal occult blood test or with high-risk findings on flexible sigmoidoscopy). The Global Rating Score quality indicators for Bowel Cancer Screening Programme colonoscopy were used as outcome measures, and trends over time were analysed. These included caecal intubation rate, adenoma detection rate, colorectal cancer detection rate, proportion of patients with minimal or mild discomfort scores, proportion of patients who required intravenous sedation, and adverse events. RESULTS: There were 5352 colonoscopies included, performed by 3 endoscopists; 73.8% were index procedures (i.e. first Bowel Cancer Screening Programme colonoscopy) and the remainder were follow-up or surveillance colonoscopies. The mean age of patients was 66 (standard deviation 5) years, and 59.8% were male. Mean age increased over time (R2=0.033; p < 0.001). There were significant trends over time towards higher caecal intubation rate (p = 0.015), higher adenoma detection rate (p < 0.001), lower proportion requiring intravenous sedation (p < 0.001). There were no significant trends in comfort scores (p = 0.606), adverse events (p = 0.503) or colorectal cancer detection (p = 0.089). CONCLUSION: There was a consistent improvement in the Global Rating Score for Bowel Cancer Screening Programme colonoscopies since the start of the programme, even when quality was already high at the start. Patients can expect high-quality colonoscopy when participating in the Bowel Cancer Screening Programme.

How narrative source impacts convergence of ratings from the Social Cognition and Object Relations Scale-Global Rating Method with psychotherapy process measures.

This study examines the construct validity of the Social Cognition and Object Relations Scale-Global Rating Method (SCORS-G; Westen, 1995; see also Stein & Mulford, 2018) by exploring the degree of convergence across different narrative sources (i.e., early memories [EM] and psychotherapy narratives [PT]) in relation to patient- and therapist-rated psychotherapy process measures. Using a university-based outpatient sample (n = 81), we found limited convergence for SCORS-G ratings across narrative type. First, paired t tests showed that the means for six of the eight SCORS-G dimensions differed significantly between the EM and PT narratives with the majority having a large magnitude of effect. Moreover, despite 29 significant correlations between a SCORS-G dimension and either an alliance or session quality variable, only two of the eight SCORS-G dimensions significantly correlated with the same process variable across narrative type (e.g., patient-rated session depth with SCORS-G Self Esteem [SE] and Identity and Coherence of Self [ICS]). Importantly, the high degree of theoretical coherence in the associations that emerged between the SCORS-G dimensions and the process variables suggest that the lack of convergence was not due to limited validity of the SCORS-G. Instead, the results underscore the importance of multi-method assessment techniques by highlighting that the manner in which a narrative is elicited will impact the object relational content patients provide. Future research and clinical implications related to the SCORS-G, alliance and psychotherapy process are discussed.